1. Field of the Invention
The present invention relates to an optical 3D digitizer with an enlarged non-ambiguity zone, having various applications for example in the field of color non-contact optical 3D digitization, computer assisted 3D vision, human body digitizing, computer animation and computer graphics, electronic games, 3D electronic archiving, 3D web, reverse engineering and 3D medical imaging.
2. Description of the Related Technology
3D digitizing, particularly non-contact optical 3D digitizing techniques, have become commercially available during recent years. Most of these techniques are based on the principle of optical triangulation. Despite the fact that passive optical triangulation (stereo vision) has been studied and used for many years for photogrammetric measurements, the active optical triangulation technique (particularly laser scanning technique) has gained popularity because of its robustness and simplicity to process data obtained using a computer. Most of the systems based on the active optical triangulation principle were developed for industrial applications, such as robotic assembly, robot guidance, industrial inspection, reverse engineering, etc.
As an example of such technique, a laser beam or a laser stripe is projected on a 3D surface of an object, scattering the laser beam or laser stripe on the surface. It is measured using a photo-electronic device. A signal can be generated indicating the position (usually the depth) of the measured point. In most cases, the basic measurements are either a point or a section profile. A mechanical or optical scanning device is usually used to provide a frame of 3D measurement. For industrial applications, mechanical scanning can be accomplished by the mechanism on which the digitizing device is mounted, such as a robot or a conveyer. The scanning process consists of a sequential data acquisition process and takes relatively longer time to scan a surface. During the scanning, the object should be kept immobilized; this is a major problem when scanning a live being. Different techniques, such as the projection of multiple stripes, laser line scanning during one video frame and high speed scanning, have been developed. These approaches are either too expensive to realize, or their sampling rate is still too low compared to 2D digital imaging.
A laser beam is a monochromatic light source. One single monochromatic laser beam cannot provide full color information of the measured surface. On the other hand, a number of today's 3D applications including computer animation, electronic games, 3D Web, 3D archiving and 3D medical imaging require information on color texture which contributes to most of the visual effects. In order to measure the color texture of a surface, a 3D digitizing system based on a laser scanning principle has to use multiple laser sources (blue, green and red lasers) or use a second camera to get color data. The first solution is very difficult to be implemented and is also very expensive. The second can suffer from problems of misalignment between 3D geometric data and color texture data because they are not captured from the same angle of the view.
Other active optical 3D digitizing techniques are based on structured light projection (pattern coding, moiré or interferometry) or random pattern projection (active stereogrammetry). For all these active optical 3D digitizing techniques, the measurement space is limited by the intersection of the usable light projection coverage (field of view and depth of field) and the usable viewing coverage (field of view and depth of field of at least one optical sensor). If the light projection pattern is periodic, the non-ambiguity zone is limited by the triangulation angle and the period. Different tricks may be used for removing the light projection pattern and get a colored texture image of the measured object. In that case, the same optical sensor may be used for measuring the 3D information and the texture of the surface of the subject, with a unique relation.
An active optical 3D digitizing technique is described in U.S. Pat. No. 6,493,095 (Song et al.). Even if this state of the art technique gives impressive results, it is still limited by the encoding points in the light projection pattern and its non-ambiguity zone that is typically much smaller than the depth of field. That may lead to partial or complete discontinuity in the unwrapped phase function if the surface of the subject (3D object) is not continuous. In most cases, a trained user can overcome these limitations; but it is difficult to get an optical 3D digitizer easy to use or even completely automated for any type of subject.
Summary of Certain Inventive Aspects of the Invention One aspect of the invention is to provide an optical 3D digitizer having an enlarged non-ambiguity zone.
Another aspect of the invention is to provide an optical 3D digitizer having nice features of light projection moiré interferometry combined to an automatic order function determination, leading to a precise, simple to use and cost-effective optical 3D digitizer.
Another aspect of the invention is to provide an optical 3D digitizer which no longer requires encoding projected lighting with information related to absolute positions in the image.
According to one aspect of the present invention, there is provided an optical 3D digitizer with an enlarged non-ambiguity zone, comprising: at least one structured light projector for projecting a fringe pattern over a target area, the fringe pattern having a shiftable position over the target area; a first camera directed toward the target area and positioned with respect to said at least one structured light projector to define a first triangulation plane therewith; a second camera directed toward the target area and positioned with respect to said at least one structured light projector to define a second triangulation plane therewith, the second triangulation plane being distinct from the first triangulation plane, the first and second cameras having at least partially overlapping measurement fields, the second camera having a larger non-ambiguity depth than the first camera; and a computer means connected to the cameras, for performing an image processing of images captured by the cameras, the image processing including evaluating a same set of camera-projector related functions from images including the pattern projected by said at least one structured light projector at shifted positions as captured by the cameras, building a low depth resolution 3D model from the camera-projector related functions evaluated with respect to the second camera, building a degenerated 3D model from the camera-projector related functions evaluated with respect to the first camera, determining chromatic texture from the images captured by the cameras, and building a complete textured 3D model from data corresponding between the low depth resolution and degenerated 3D models within a tolerance range.
According to another aspect of the present invention, there is also provided an optical 3D digitizing method with an enlarged non-ambiguity zone, comprising: controllably projecting a fringe pattern over a target area using at least one structured light projector, the fringe pattern having a shiftable position over the target area; positioning a first camera directed toward the target area with respect to said at least one structured light projector to define a first triangulation plane therewith; positioning a second camera directed toward the target area with respect to said at least one structured light projector to define a second triangulation plane therewith, the second triangulation plane being distinct from the first triangulation plane, the first and second cameras having at least partially overlapping measurement fields, the second camera having a larger non-ambiguity depth than the first camera; and performing an image processing of images captured by the cameras, the image processing including evaluating a same set of camera-projector related functions from images including the pattern projected by said at least one structured light projector at shifted positions as captured by the cameras, building a low depth resolution 3D model from the camera-projector related functions evaluated with respect to the second camera, building a degenerated 3D model from the camera-projector related functions evaluated with respect to the first camera, determining chromatic texture from the images captured by the cameras, and building a complete textured 3D model from data corresponding between the low depth resolution and degenerated 3D models within a tolerance range.
According to another aspect of the present invention, there is also provided a computer apparatus for performing an image processing of images captured by first and second cameras, the second camera having a larger non-ambiguity depth than the first camera, comprising: means for evaluating a same set of camera-projector related functions from images captured by the cameras, at least some of the images including a pattern projected at shifted positions; means for building a low depth resolution 3D model from the camera-projector related functions evaluated with respect to the second camera; means for building a degenerated 3D model from the camera-projector related functions evaluated with respect to the first camera; means for determining chromatic texture from the images captured by the cameras; and means for building a complete textured 3D model from data corresponding between the low depth resolution and degenerated 3D models within a tolerance range.
According to another aspect of the present invention, there is also provided a computer readable medium having recorded thereon statements and instructions for execution by a computer to perform an image processing of images captured by first and second cameras directed toward a target area, the second camera having a larger non-ambiguity depth than the first camera, the image processing including evaluating a same set of camera-projector related functions from the images captured by the cameras, at least some of the images including a pattern projected at shifted positions, building a low depth resolution 3D model from the camera-projector related functions evaluated with respect to the second camera, building a degenerated 3D model from the camera-projector related functions evaluated with respect to the first camera, determining chromatic texture from the images captured by the cameras, and building a complete textured 3D model from data corresponding between the low depth resolution and degenerated 3D models within a tolerance range.
According to another aspect of the present invention, there is also provided a computer program product, comprising a memory having computer readable code embodied therein, for execution by a CPU, for performing an image processing of images captured by first and second cameras directed toward a target area, the second camera having a larger non-ambiguity depth than the first camera, said code comprising: code means for evaluating a same set of camera-projector related functions from the images captured by the cameras, at least some of the images including a pattern projected at shifted positions; code means for building a low depth resolution 3D model from the camera-projector related functions evaluated with respect to the second camera; code means for building a degenerated 3D model from the camera-projector related functions evaluated with respect to the first camera; code means for determining chromatic texture from the images captured by the cameras; and code means for building a complete textured 3D model from data corresponding between the low depth resolution and degenerated 3D models within a tolerance range.
According to another aspect of the present invention, there is also provided a carrier wave embodying a computer data signal representing sequences of statements and instructions which, when executed by a processor, cause the processor to perform an image processing of images captured by first and second cameras directed toward a target area, the second camera having a larger non-ambiguity depth than the first camera, the statements and instructions comprising: evaluating a same set of camera-projector related functions from the images captured by the cameras, at least some of the images including a pattern projected at shifted positions; building a low depth resolution 3D model from the camera-projector related functions evaluated with respect to the second camera; building a degenerated 3D model from the camera-projector related functions evaluated with respect to the first camera; determining chromatic texture from the images captured by the cameras; and building a complete textured 3D model from data corresponding between the low depth resolution and degenerated 3D models within a tolerance range.
According to another aspect of the present invention, there is also provided an optical 3D digitizing method with an enlarged non-ambiguity zone, comprising: controllably projecting a fringe pattern having a shiftable position over a target area; capturing images obtained by high depth resolution sensing and low depth resolution sensing from respective measurement fields at least partially overlapping each other over the target area; determining absolute pixel 3D positions in the images obtained by low depth resolution sensing and high depth resolution sensing as a function of relations depending on the fringe pattern in the captured images and correspondence between the absolute pixel 3D positions in the images; extracting chromatic texture from the captured images; and building a complete textured 3D model from the absolute pixel 3D positions and the chromatic texture.
The following provides a non-restrictive outline of certain possibly preferably features of the invention which will be more fully described hereinafter.
The optical 3D digitizer may consist of two color (color version) or monochromatic (B/W version) cameras and a light projection system. Each camera-projector couple is defined by a triangulation plane and a usable coverage (field of view and depth of field). The two couples of camera-projector increase the mathematical constrains and remove the ambiguity of a periodicity of a phase function by evaluating an order function. That leads to an optical 3D digitizing apparatus with a composed non-ambiguity zone extended to fill the entire usable coverage (field of view and depth of field).
A possible configuration is based on two (or more) synchronized cameras and one white light projected pattern, defining two optical triangulation planes. The two planes may be coplanar or show a defined relative angle, and their respective usable coverage are overlapping. The two baselines, the angle between the triangulation planes and the orientation of the projected pattern, allow adjustment of two different depth sensitivities and non-ambiguity zones. It is then possible to get a low sensitivity camera-projector couple with a large non-ambiguity zone for the full usable coverage, and a high sensitivity camera-projector couple with a small non-ambiguity zone leading to multiple orders in its phase function, thus giving a higher resolution for the depth. If the two optical triangulation planes are not coplanar, it is easy with a slight tilt of a projection fringe pattern to get a ratio of non-ambiguity depth greater than ten. Correspondence between the two cameras is defined by the configuration and represented by a look-up table (LUT). Once the phase and order functions are evaluated for all the cameras, a 3D model can be automatically constructed using a LUT. An optical device may be used to remove the structures in the projected pattern leading to homogeneous white lighting. This lighting or other external lighting may be used for measuring the colored (or chromatic) texture of a subject, with the same cameras. The colored texture is then naturally and exactly mapped onto the 3D model. It is also possible to evaluate the color information of the subject from the images containing the projected pattern. The 3D information and generated texture can be extracted using mathematical functions (such as the phase, phase shift, average and modulation functions) for one or multiple projected pattern images of the subject, as seen by the cameras. All these functions are defined for every pixel, leading to the highest possible resolution. Using synchronized cameras allows reaching the shortest acquisition time for the 3D digitizing device as if there were only one camera. The overall acquisition time will be a fraction of a second with commercially available standard cameras: such an acquisition time is critical for efficient 3D digitizing, especially for living subject.
Each of the camera-projector couple has the capacity of measuring many mathematical functions for every individual pixels of the camera. By analyzing these functions for all of the camera's overlapped fields of view (such as the phase, phase shift, average and modulation functions), it is possible to increase the non-ambiguity zone, up to cover all the effective measurement volume. The number of image pixels captured by each camera is limited either by industrial standards (for example, NTSC or PAL) or by the manufacturing cost of the camera. This limitation does not apply to the projected pattern. In fact, the only limit for a projected pattern (a film, for example) is the optical resolution of the film and projection optics. It is not uncommon to obtain a resolution of 50 to 100 lines per mm on a pattern to be projected, which may have a size of 35 mm×25 mm. So it is evident that one projected pattern can easily provide the necessary image information for the area covered by at least two cameras.
One approach for 3D coordinate measurements may use one video frame containing a projected fringe pattern. Another approach may require a few video images, which also contain a projected fringe pattern. An image processing based on analysis of mechanical interference pattern provides the 3D coordinate data for each image pixel. So one or more combined images are required to provide 3D measurement of one view of a subject. The acquisition time of one view requires a fraction of a second using commercially available standard cameras. The necessary acquisition time of the digitizer is much shorter than most of existing techniques based on laser scanning principles. Also, many more data points can be measured on a living 3D object who does not need special training to be kept immobilized for several seconds.
The cameras used for the measurement of 3D geometry may provide directly the capturing of color or gray scale texture. In order to ensure a uniform illumination during the capturing of texture, a defocusing optical element which removes the fringe pattern from the image may be used. Another possibility is to block the projected light and use the surrounding or auxiliary lighting for the acquisition of the texture image. It is also possible to evaluate the texture image from known algorithm related to the fringe images. In all these cases, since the same image pixel of the camera measures the 3D geometry and texture data of a point on a 3D surface, the texturing mapping on top of 3D geometry is automatically ensured by the nature of this data acquisition. The 3D digitizer does not need two image sensors to separately measure 3D geometry and texture and avoids the problem of misalignment occurring with laser scanning systems.
The data of 3D coordinates obtained from at least one camera is preferably calibrated in an orthonormal coordinate space. An analysis using the mathematical functions from all the cameras (such as the phase, phase shift, average and modulation functions) allows determination of the order of the phase function for at least one camera. The result is at least one fully textured and accurate 3D model ready to use. The relationship between all these mathematical functions may be defined experimentally during a calibration process that is performed under known conditions. This relationship ensures the conversion of the measurement in computer unit to real physical parameters for the whole surface of the subject to measure in 3D.